Structural Aggregation Invariance (SAI)

If we compare the absolute deviation across method pairs and whether or not SAI holds for a model, the following plot shows that for models for which SAI does not hold (i.e., the pair clustering model) the absolute deviation is on average larger. The red points is the mean in each plot. Consequently, all following analyses are based on the data excluding the non SAI (i.e., pair clustering) model.

Empirical Aggregation Invariance (EAI)

Reference Method: CP-MLE

Selected Pairs: SE < .025, Rho < .1

model2 dataset parameter cond_x cond_y abs_dev se_c rhos_max x y prop_ns_noasy prop_ns_trait prop_ns_nopb prop_ns_nonpb prop_ns z_abs_dev
hb Bayen2006Exp1.csv:hb hb:c NP-BAYES CP-MLE 0.189 0.021 0.034 0.200 0.011 0.038 0.000 0.038 0.000 0.019 8.915
pm Smith & Hunt 2013.csv:pm pm:M NP-BAYES CP-MLE 0.159 0.019 0.081 0.831 0.991 0.000 0.000 0.200 0.160 0.090 8.501
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-BAYES CP-MLE 0.091 0.013 0.025 0.390 0.300 0.182 0.101 0.141 0.000 0.106 7.065
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-BAYES CP-MLE 0.053 0.009 0.047 0.070 0.018 0.455 0.030 0.091 0.030 0.152 5.906
quad CalanchiniEtAl2014_PI_ableMCMC:quad quad:D1 PP-LT-NC CP-MLE 0.035 0.007 0.091 0.887 0.852 0.230 0.060 0.190 0.025 0.126 5.028
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r PP-B CP-MLE 0.106 0.021 0.025 0.406 0.300 0.182 0.101 0.141 0.000 0.106 4.994
pm Smithetal2014_Exp2.csv:pm pm:C2 NP-BAYES CP-MLE 0.059 0.013 0.071 0.837 0.895 0.000 0.000 0.067 0.022 0.022 4.400
rm Pohl et al 2017.csv:rm rm:r NP-BAYES CP-MLE 0.039 0.010 0.007 0.789 0.828 0.012 0.012 0.062 0.025 0.028 3.815
rm Pohl et al 2017.csv:rm rm:r PP-LT-NC CP-MLE 0.054 0.016 0.007 0.882 0.828 0.012 0.012 0.062 0.025 0.028 3.455
rm Pohl et al 2017.csv:rm rm:r PP-LT-C CP-MLE 0.053 0.016 0.007 0.881 0.828 0.012 0.012 0.062 0.025 0.028 3.398
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-MLE CP-MLE 0.078 0.024 0.025 0.377 0.300 0.182 0.101 0.141 0.000 0.106 3.177
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-PB CP-MLE 0.078 0.024 0.025 0.377 0.300 0.182 0.101 0.141 0.000 0.106 3.177
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-NPB CP-MLE 0.078 0.024 0.025 0.377 0.300 0.182 0.101 0.141 0.000 0.106 3.177
hb Bayen2006Exp1.csv:hb hb:c PP-B CP-MLE 0.017 0.005 0.034 0.028 0.011 0.038 0.000 0.038 0.000 0.019 3.095
rm HilbigPohl09exp3.csv:rm rm:r NP-BAYES CP-MLE 0.044 0.015 0.086 0.716 0.760 0.048 0.016 0.032 0.000 0.024 2.956
rm Horn Ruggeri Pachur 2016 .csv:rm rm:r NP-BAYES CP-MLE 0.066 0.024 0.086 0.327 0.262 0.167 0.056 0.111 0.000 0.083 2.754
quad CalanchiniEtAl2014_PI_ableMCMC:quad quad:D1 NP-MLE CP-MLE 0.020 0.007 0.091 0.872 0.852 0.230 0.060 0.190 0.025 0.126 2.754
quad CalanchiniEtAl2014_PI_ableMCMC:quad quad:D1 NP-PB CP-MLE 0.020 0.007 0.091 0.872 0.852 0.230 0.060 0.190 0.025 0.126 2.731
quad CalanchiniEtAl2014_PI_ableMCMC:quad quad:D1 NP-NPB CP-MLE 0.020 0.007 0.091 0.872 0.852 0.230 0.060 0.190 0.025 0.126 2.731
quad JinEtAl2016s2:quad quad:ACbb1 NP-BAYES CP-MLE 0.046 0.017 0.095 0.133 0.086 0.130 0.000 0.000 0.000 0.033 2.697
rm Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm rm:r NP-BAYES CP-MLE 0.030 0.011 0.035 0.574 0.604 0.071 0.048 0.071 0.000 0.048 2.636
quad GonsalkoraleEtAl2011:quad quad:ACbb1 NP-BAYES CP-MLE 0.046 0.019 0.095 0.136 0.089 0.174 0.043 0.000 0.000 0.054 2.467
pm Smith & Hunt 2013.csv:pm pm:M PP-B CP-MLE 0.030 0.012 0.081 0.961 0.991 0.000 0.000 0.200 0.160 0.090 2.416
pm Smithetal2014_Exp2.csv:pm pm:C1 NP-BAYES CP-MLE 0.045 0.019 0.095 0.664 0.710 0.000 0.000 0.067 0.044 0.028 2.342
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-B CP-MLE 0.024 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.300
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-MLE CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-PB CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-NPB CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
rm Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm rm:r NP-BAYES CP-MLE 0.016 0.009 0.019 0.606 0.622 0.172 0.086 0.155 0.000 0.103 1.894
rm HilbigPohl09exp2.csv:rm rm:a NP-BAYES CP-MLE 0.011 0.006 0.066 0.785 0.796 0.097 0.042 0.097 0.014 0.062 1.885

Selected Pairs: SE < .02, Rho < .05

model2 dataset parameter cond_x cond_y abs_dev se_c rhos_max x y prop_ns_noasy prop_ns_trait prop_ns_nopb prop_ns_nonpb prop_ns z_abs_dev
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-BAYES CP-MLE 0.091 0.013 0.025 0.390 0.300 0.182 0.101 0.141 0.000 0.106 7.065
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-BAYES CP-MLE 0.053 0.009 0.047 0.070 0.018 0.455 0.030 0.091 0.030 0.152 5.906
rm Pohl et al 2017.csv:rm rm:r NP-BAYES CP-MLE 0.039 0.010 0.007 0.789 0.828 0.012 0.012 0.062 0.025 0.028 3.815
rm Pohl et al 2017.csv:rm rm:r PP-LT-NC CP-MLE 0.054 0.016 0.007 0.882 0.828 0.012 0.012 0.062 0.025 0.028 3.455
rm Pohl et al 2017.csv:rm rm:r PP-LT-C CP-MLE 0.053 0.016 0.007 0.881 0.828 0.012 0.012 0.062 0.025 0.028 3.398
hb Bayen2006Exp1.csv:hb hb:c PP-B CP-MLE 0.017 0.005 0.034 0.028 0.011 0.038 0.000 0.038 0.000 0.019 3.095
rm Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm rm:r NP-BAYES CP-MLE 0.030 0.011 0.035 0.574 0.604 0.071 0.048 0.071 0.000 0.048 2.636
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-B CP-MLE 0.024 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.300
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-MLE CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-PB CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-NPB CP-MLE 0.025 0.011 0.047 0.042 0.018 0.455 0.030 0.091 0.030 0.152 2.160
rm Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm rm:r NP-BAYES CP-MLE 0.016 0.009 0.019 0.606 0.622 0.172 0.086 0.155 0.000 0.103 1.894
quad JinEtAl2016s2:quad quad:ACbb1 NP-BAYES CP-MLE 0.031 0.017 0.044 0.156 0.126 0.034 0.000 0.000 0.000 0.009 1.809
pd_s stahl_2015_exp1.csv:pd_s pd_s:A NP-BAYES CP-MLE 0.010 0.006 0.047 0.244 0.254 0.455 0.030 0.091 0.030 0.152 1.695
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-BAYES CP-MLE 0.031 0.019 0.035 0.031 0.000 0.882 0.765 0.824 0.000 0.618 1.668
rm HilbigPohl09exp2.csv:rm rm:b NP-BAYES CP-MLE 0.012 0.007 0.029 0.671 0.684 0.097 0.042 0.097 0.014 0.062 1.662
rm Michalkiewicz Erdfelder 2016_Exp.2_week group_test2.csv:rm rm:r NP-BAYES CP-MLE 0.015 0.009 0.028 0.713 0.728 0.190 0.071 0.190 0.000 0.113 1.634
pm SmithBayen2005_Experiment2.csv:pm pm:C1 NP-BAYES CP-MLE 0.025 0.020 0.030 0.834 0.860 0.000 0.000 0.000 0.000 0.000 1.302
2htsm_4 S2002_E2:2htsm_4 2htsm_4:D NP-BAYES CP-MLE 0.015 0.013 0.042 0.904 0.919 0.000 0.000 0.000 0.000 0.000 1.137
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-LT-NC CP-MLE 0.010 0.009 0.047 0.008 0.018 0.455 0.030 0.091 0.030 0.152 1.111
rm Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm rm:a NP-BAYES CP-MLE 0.006 0.006 0.043 0.528 0.522 0.172 0.086 0.155 0.000 0.103 1.010
hb Bayen2006Exp1.csv:hb hb:c NP-MLE CP-MLE 0.004 0.004 0.034 0.008 0.011 0.038 0.000 0.038 0.000 0.019 1.003
hb Bayen2006Exp1.csv:hb hb:c NP-PB CP-MLE 0.004 0.004 0.034 0.008 0.011 0.038 0.000 0.038 0.000 0.019 1.003
hb Bayen2006Exp1.csv:hb hb:c NP-NPB CP-MLE 0.004 0.004 0.034 0.008 0.011 0.038 0.000 0.038 0.000 0.019 1.003
rm HilbigSchollPohl2010exp2.csv:rm rm:r PP-B CP-MLE 0.016 0.017 0.035 0.016 0.000 0.882 0.765 0.824 0.000 0.618 0.933
rm Hilbig et al 2015_exp3.csv:rm rm:a NP-BAYES CP-MLE 0.013 0.014 0.017 0.690 0.703 0.062 0.000 0.062 0.000 0.031 0.932
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-LT-C CP-MLE 0.008 0.009 0.047 0.009 0.018 0.455 0.030 0.091 0.030 0.152 0.928
rm HilbigPohl09exp2.csv:rm rm:b NP-MLE CP-MLE 0.008 0.009 0.029 0.675 0.684 0.097 0.042 0.097 0.014 0.062 0.905
rm HilbigPohl09exp2.csv:rm rm:b NP-PB CP-MLE 0.008 0.009 0.029 0.675 0.684 0.097 0.042 0.097 0.014 0.062 0.905
rm HilbigPohl09exp2.csv:rm rm:b NP-NPB CP-MLE 0.008 0.009 0.029 0.675 0.684 0.097 0.042 0.097 0.014 0.062 0.905

Reference Method:PP-LT-C

Selected Pairs: SE < .025, Rho < .1

model2 dataset parameter cond_x cond_y abs_dev se_c rhos_max x y prop_ns_noasy prop_ns_trait prop_ns_nopb prop_ns_nonpb prop_ns z_abs_dev
hb Coolin2015.csv:hb hb:c NP-BAYES PP-LT-C 0.246 0.016 0.061 0.248 0.002 0.047 0.031 0.094 0.000 0.043 15.572
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-BAYES PP-LT-C 0.061 0.007 0.047 0.070 0.009 0.455 0.030 0.091 0.030 0.152 8.282
pm Smith & Hunt 2013.csv:pm pm:M NP-BAYES PP-LT-C 0.154 0.020 0.081 0.831 0.985 0.000 0.000 0.200 0.160 0.090 7.636
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-BAYES PP-LT-C 0.030 0.005 0.035 0.031 0.001 0.882 0.765 0.824 0.000 0.618 6.217
rm Pohl et al 2017.csv:rm rm:r NP-BAYES PP-LT-C 0.092 0.016 0.007 0.789 0.881 0.012 0.012 0.062 0.025 0.028 5.929
rm HilbigSchollPohl2010exp2.csv:rm rm:r PP-B PP-LT-C 0.015 0.003 0.035 0.016 0.001 0.882 0.765 0.824 0.000 0.618 4.708
pm Smithetal2014_Exp2.csv:pm pm:C2 NP-BAYES PP-LT-C 0.066 0.015 0.071 0.837 0.902 0.000 0.000 0.067 0.022 0.022 4.365
hb Coolin2015.csv:hb hb:c PP-B PP-LT-C 0.016 0.004 0.061 0.018 0.002 0.047 0.031 0.094 0.000 0.043 4.097
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-B PP-LT-C 0.033 0.009 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.624
rm HilbigPohl09exp3.csv:rm rm:r NP-BAYES PP-LT-C 0.076 0.021 0.086 0.716 0.792 0.048 0.016 0.032 0.000 0.024 3.559
rm Pohl et al 2017.csv:rm rm:r CP-Bayes PP-LT-C 0.054 0.016 0.007 0.827 0.881 0.012 0.012 0.062 0.025 0.028 3.425
rm Pohl et al 2017.csv:rm rm:r CP-MLE PP-LT-C 0.053 0.016 0.007 0.828 0.881 0.012 0.012 0.062 0.025 0.028 3.398
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-MLE PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-PB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-NPB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
rm Pohl et al 2017.csv:rm rm:r NP-PB PP-LT-C 0.061 0.021 0.007 0.820 0.881 0.012 0.012 0.062 0.025 0.028 2.873
rm Pohl et al 2017.csv:rm rm:r NP-NPB PP-LT-C 0.061 0.021 0.007 0.820 0.881 0.012 0.012 0.062 0.025 0.028 2.873
rm Pohl et al 2017.csv:rm rm:r NP-MLE PP-LT-C 0.062 0.021 0.007 0.820 0.881 0.012 0.012 0.062 0.025 0.028 2.873
quad JinEtAl2016s2:quad quad:ACbb1 NP-BAYES PP-LT-C 0.051 0.018 0.095 0.133 0.082 0.130 0.000 0.000 0.000 0.033 2.873
quad GonsalkoraleEtAl2011:quad quad:ACbb1 NP-BAYES PP-LT-C 0.062 0.022 0.095 0.136 0.074 0.174 0.043 0.000 0.000 0.054 2.872
rm Pohl et al 2017.csv:rm rm:r PP-B PP-LT-C 0.056 0.020 0.007 0.825 0.881 0.012 0.012 0.062 0.025 0.028 2.836
pm Smithetal2014_Exp2.csv:pm pm:C1 NP-BAYES PP-LT-C 0.058 0.024 0.095 0.664 0.722 0.000 0.000 0.067 0.044 0.028 2.384
hb Coolin2015.csv:hb hb:c NP-PB PP-LT-C 0.014 0.007 0.061 0.016 0.002 0.047 0.031 0.094 0.000 0.043 2.126
hb Coolin2015.csv:hb hb:c NP-NPB PP-LT-C 0.014 0.007 0.061 0.016 0.002 0.047 0.031 0.094 0.000 0.043 2.126
quad JinEtAl2016s2:quad quad:ACbb1 NP-BAYES PP-LT-C 0.035 0.018 0.044 0.156 0.121 0.034 0.000 0.000 0.000 0.009 1.988
rm HilbigPohl09exp2.csv:rm rm:a NP-BAYES PP-LT-C 0.014 0.007 0.066 0.785 0.799 0.097 0.042 0.097 0.014 0.062 1.916
hb Coolin2015.csv:hb hb:c NP-MLE PP-LT-C 0.002 0.001 0.061 0.000 0.002 0.047 0.031 0.094 0.000 0.043 1.794
rm Michalkiewicz Erdfelder 2016_Exp.2_week group_test2.csv:rm rm:r NP-BAYES PP-LT-C 0.042 0.023 0.028 0.713 0.755 0.190 0.071 0.190 0.000 0.113 1.782
pm Smith & Hunt 2013.csv:pm pm:M PP-B PP-LT-C 0.024 0.014 0.081 0.961 0.985 0.000 0.000 0.200 0.160 0.090 1.770
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-MLE PP-LT-C 0.001 0.001 0.035 0.000 0.001 0.882 0.765 0.824 0.000 0.618 1.755

Selected Pairs: SE < .02, Rho < .05

model2 dataset parameter cond_x cond_y abs_dev se_c rhos_max x y prop_ns_noasy prop_ns_trait prop_ns_nopb prop_ns_nonpb prop_ns z_abs_dev
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-BAYES PP-LT-C 0.061 0.007 0.047 0.070 0.009 0.455 0.030 0.091 0.030 0.152 8.282
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-BAYES PP-LT-C 0.030 0.005 0.035 0.031 0.001 0.882 0.765 0.824 0.000 0.618 6.217
rm Pohl et al 2017.csv:rm rm:r NP-BAYES PP-LT-C 0.092 0.016 0.007 0.789 0.881 0.012 0.012 0.062 0.025 0.028 5.929
rm HilbigSchollPohl2010exp2.csv:rm rm:r PP-B PP-LT-C 0.015 0.003 0.035 0.016 0.001 0.882 0.765 0.824 0.000 0.618 4.708
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-B PP-LT-C 0.033 0.009 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.624
rm Pohl et al 2017.csv:rm rm:r CP-Bayes PP-LT-C 0.054 0.016 0.007 0.827 0.881 0.012 0.012 0.062 0.025 0.028 3.425
rm Pohl et al 2017.csv:rm rm:r CP-MLE PP-LT-C 0.053 0.016 0.007 0.828 0.881 0.012 0.012 0.062 0.025 0.028 3.398
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-MLE PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-PB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-NPB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
rm Pohl et al 2017.csv:rm rm:r PP-B PP-LT-C 0.056 0.020 0.007 0.825 0.881 0.012 0.012 0.062 0.025 0.028 2.836
quad JinEtAl2016s2:quad quad:ACbb1 NP-BAYES PP-LT-C 0.035 0.018 0.044 0.156 0.121 0.034 0.000 0.000 0.000 0.009 1.988
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-MLE PP-LT-C 0.001 0.001 0.035 0.000 0.001 0.882 0.765 0.824 0.000 0.618 1.755
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-PB PP-LT-C 0.001 0.001 0.035 0.000 0.001 0.882 0.765 0.824 0.000 0.618 1.755
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-NPB PP-LT-C 0.001 0.001 0.035 0.000 0.001 0.882 0.765 0.824 0.000 0.618 1.755
pd_s stahl_2015_exp1.csv:pd_s pd_s:C CP-Bayes PP-LT-C 0.010 0.009 0.047 0.019 0.009 0.455 0.030 0.091 0.030 0.152 1.113
rm HilbigPohl09exp2.csv:rm rm:b NP-BAYES PP-LT-C 0.010 0.010 0.029 0.671 0.681 0.097 0.042 0.097 0.014 0.062 1.009
pd_s stahl_2015_exp1.csv:pd_s pd_s:C CP-MLE PP-LT-C 0.008 0.009 0.047 0.018 0.009 0.455 0.030 0.091 0.030 0.152 0.928
rm Hilbig et al 2015_exp3.csv:rm rm:a NP-BAYES PP-LT-C 0.013 0.014 0.017 0.690 0.703 0.062 0.000 0.062 0.000 0.031 0.888
pd_s stahl_2015_exp1.csv:pd_s pd_s:A CP-MLE PP-LT-C 0.008 0.009 0.047 0.254 0.246 0.455 0.030 0.091 0.030 0.152 0.853
2htsm_4 S2002_E2:2htsm_4 2htsm_4:D NP-BAYES PP-LT-C 0.014 0.017 0.042 0.904 0.918 0.000 0.000 0.000 0.000 0.000 0.848
pd_s stahl_2015_exp1.csv:pd_s pd_s:A CP-Bayes PP-LT-C 0.008 0.009 0.047 0.254 0.246 0.455 0.030 0.091 0.030 0.152 0.843
quad JinEtAl2016s2:quad quad:ACbb1 NP-PB PP-LT-C 0.012 0.018 0.044 0.133 0.121 0.034 0.000 0.000 0.000 0.009 0.673
quad JinEtAl2016s2:quad quad:ACbb1 NP-NPB PP-LT-C 0.012 0.018 0.044 0.133 0.121 0.034 0.000 0.000 0.000 0.009 0.673
rm HilbigPohl2008exp5.csv:rm rm:a NP-BAYES PP-LT-C 0.006 0.010 0.041 0.588 0.595 0.059 0.020 0.059 0.000 0.035 0.596
quad JinEtAl2016s2:quad quad:ACbb1 NP-MLE PP-LT-C 0.010 0.018 0.044 0.131 0.121 0.034 0.000 0.000 0.000 0.009 0.564
rm Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm rm:a CP-MLE PP-LT-C 0.005 0.008 0.043 0.522 0.527 0.172 0.086 0.155 0.000 0.103 0.542
rm Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm rm:a CP-Bayes PP-LT-C 0.005 0.009 0.043 0.522 0.527 0.172 0.086 0.155 0.000 0.103 0.540
rm HilbigPohl09exp2.csv:rm rm:b NP-MLE PP-LT-C 0.006 0.012 0.029 0.675 0.681 0.097 0.042 0.097 0.014 0.062 0.513
rm HilbigPohl09exp2.csv:rm rm:b NP-PB PP-LT-C 0.006 0.012 0.029 0.675 0.681 0.097 0.042 0.097 0.014 0.062 0.513

Selected Pairs: SE < .05, Rho < .1

model2 dataset parameter cond_x cond_y abs_dev se_c rhos_max x y prop_ns_noasy prop_ns_trait prop_ns_nopb prop_ns_nonpb prop_ns z_abs_dev
hb Coolin2015.csv:hb hb:c NP-BAYES PP-LT-C 0.246 0.016 0.061 0.248 0.002 0.047 0.031 0.094 0.000 0.043 15.572
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-BAYES PP-LT-C 0.061 0.007 0.047 0.070 0.009 0.455 0.030 0.091 0.030 0.152 8.282
2htsm_5d Mieth et al. (2016) Ex2:2htsm_5d 2htsm_5d:d_2 NP-BAYES PP-LT-C 0.356 0.045 0.058 0.437 0.080 0.281 0.010 0.042 0.000 0.083 7.888
pm Smith & Hunt 2013.csv:pm pm:M NP-BAYES PP-LT-C 0.154 0.020 0.081 0.831 0.985 0.000 0.000 0.200 0.160 0.090 7.636
2htsm_6e Bell et al. (2015) Ex2:2htsm_6e 2htsm_6e:d_1 NP-BAYES PP-LT-C 0.351 0.047 0.084 0.448 0.097 0.982 0.009 0.026 0.000 0.254 7.530
2htsm_4 M2003:2htsm_4 2htsm_4:d NP-BAYES PP-LT-C 0.269 0.038 0.069 0.321 0.052 0.061 0.000 0.030 0.000 0.023 7.093
2htsm_4 M2003:2htsm_4 2htsm_4:d NP-BAYES PP-LT-C 0.259 0.041 0.070 0.327 0.068 0.061 0.000 0.030 0.000 0.023 6.334
rm HilbigSchollPohl2010exp2.csv:rm rm:r NP-BAYES PP-LT-C 0.030 0.005 0.035 0.031 0.001 0.882 0.765 0.824 0.000 0.618 6.217
rm Pohl et al 2017.csv:rm rm:r NP-BAYES PP-LT-C 0.092 0.016 0.007 0.789 0.881 0.012 0.012 0.062 0.025 0.028 5.929
2htsm_4 MH2001:2htsm_4 2htsm_4:d NP-BAYES PP-LT-C 0.208 0.039 0.079 0.253 0.045 0.100 0.000 0.000 0.000 0.025 5.338
2htsm_4 MH2001:2htsm_4 2htsm_4:d NP-BAYES PP-LT-C 0.228 0.044 0.077 0.285 0.057 0.100 0.000 0.000 0.000 0.025 5.237
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-BAYES PP-LT-C 0.161 0.033 0.025 0.390 0.229 0.182 0.101 0.141 0.000 0.106 4.878
2htsm_4 BK2011_E1:2htsm_4 2htsm_4:b NP-BAYES PP-LT-C 0.227 0.047 0.084 0.330 0.103 0.042 0.000 0.083 0.000 0.031 4.790
rm HilbigSchollPohl2010exp2.csv:rm rm:r PP-B PP-LT-C 0.015 0.003 0.035 0.016 0.001 0.882 0.765 0.824 0.000 0.618 4.708
pm Smithetal2014_Exp2.csv:pm pm:C2 NP-BAYES PP-LT-C 0.066 0.015 0.071 0.837 0.902 0.000 0.000 0.067 0.022 0.022 4.365
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r PP-B PP-LT-C 0.176 0.041 0.025 0.406 0.229 0.182 0.101 0.141 0.000 0.106 4.260
hb Coolin2015.csv:hb hb:c PP-B PP-LT-C 0.016 0.004 0.061 0.018 0.002 0.047 0.031 0.094 0.000 0.043 4.097
hb GroßBayen2015.csv:hb hb:b NP-BAYES PP-LT-C 0.113 0.030 0.048 0.299 0.412 0.022 0.022 0.022 0.000 0.016 3.696
pd_s stahl_2015_exp1.csv:pd_s pd_s:C PP-B PP-LT-C 0.033 0.009 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.624
rm HilbigPohl09exp3.csv:rm rm:r NP-BAYES PP-LT-C 0.076 0.021 0.086 0.716 0.792 0.048 0.016 0.032 0.000 0.024 3.559
rm Pohl et al 2017.csv:rm rm:r CP-Bayes PP-LT-C 0.054 0.016 0.007 0.827 0.881 0.012 0.012 0.062 0.025 0.028 3.425
rm Pohl et al 2017.csv:rm rm:r CP-MLE PP-LT-C 0.053 0.016 0.007 0.828 0.881 0.012 0.012 0.062 0.025 0.028 3.398
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-MLE PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-PB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
pd_s stahl_2015_exp1.csv:pd_s pd_s:C NP-NPB PP-LT-C 0.033 0.010 0.047 0.042 0.009 0.455 0.030 0.091 0.030 0.152 3.350
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-MLE PP-LT-C 0.148 0.045 0.025 0.377 0.229 0.182 0.101 0.141 0.000 0.106 3.319
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-PB PP-LT-C 0.148 0.045 0.025 0.377 0.229 0.182 0.101 0.141 0.000 0.106 3.319
rm Data from Filevich Horn Kuehn 2017_within2.csv:rm rm:r NP-NPB PP-LT-C 0.148 0.045 0.025 0.377 0.229 0.182 0.101 0.141 0.000 0.106 3.319
rm Pohl et al 2017.csv:rm rm:r NP-PB PP-LT-C 0.061 0.021 0.007 0.820 0.881 0.012 0.012 0.062 0.025 0.028 2.873
rm Pohl et al 2017.csv:rm rm:r NP-NPB PP-LT-C 0.061 0.021 0.007 0.820 0.881 0.012 0.012 0.062 0.025 0.028 2.873